The present invention relates to methods and devices for generating multispectral or hyperspectral illuminating light having an addressable spectrum, in particular for (adaptive) multispectral or hyperspectral imaging, for detecting structural or topographic object information, and/or for measuring the two-dimensional (2D) or three-dimensional (3D) profile of an object, or for measuring the distance to an object by means of spectrometry
Document DE 10 2006 007 172 B4 discloses a method and a device for rapid, space-resolved, two-dimensional spectroscopic analysis and multispectral imaging. The device comprises a rasterized, two-dimensional array of micro-lenses and pinholes assigned to the respective focal points of the micro-lenses, a two-dimensional detector matrix in the detection light path, and means for lateral spectral splitting and for focusing the multispectral light incident on the two-dimensional detector matrix such that a spectral axis is present on the detector matrix. The two-dimensional array is arranged on the two-dimensional matrix so as to be inclined toward the spectral axis by an acute angle, thereby allowing for optimum use of the detector matrix area. Space resolution of this method, however, is not optimum.
Document U.S. Pat. No. 8,014,569 B2 discloses a method and a device for food assessment. In this method, light patterns generated by means of DMD application are projected onto the sample (fruit, vegetable or other food), said light patterns being subsequently analysed by means of variable spectral filters so as to obtain information on the state of the sample.
The article “Snapshot Hyperspectral Imaging in Ophthalmology” by W. R. Johnson et al., published in Journal of Biomedical Optics Vol 12 (1) 014036, 2007, discloses a device for hyperspectral imaging. The spectral data are obtained from a single camera image only, which is why comparatively high demands must be made on the grayscale resolution of the camera in order to be able to achieve acceptable color resolution. Therefore, the solution presented in this article can be very limited as regards photometric accuracy.
The article “Development of a digital-micromirror-device-based multishot snapshot spectral imaging system” by Y. Wu et al., published in Optics Letters, Vol. 36, No. 14, pp 2692-1694, 2011 likewise presents a solution including only one camera image for spectral imaging with compressive sensing (CS). The results shown do not exhibit high photometric accuracy, which is inherent to the principle as only one camera image is processed. The required processing power, however, is comparatively high, when standard computer technology in the year 2010 and a typical color object are considered.
The article “Dispersive interferometric profilometer” by J. Schwider and L. Zhou, published in Optics Letters, Vol. 19, No. 13, pp 995-997, 1994 suggests an interferometric system which combines a two-beam interferometer and a spectrometer. Therein, the interference signal is spectrally split by means of a grating such that stripes of the same color order are generated. Two-dimensional detection of the measured object can only be achieved in a time-serial manner by relative movement between spectrometer and measured object.
The article “Multi-frequency Light Source Using Spatial Light Modulator for Profilometry”, S. Choi et al., published in the Conference Proceedings of CLEO-PR 2013, WPF-15, suggests a light source for generating spectral frequency combs with controllable wavelengths and frequency spacing for spectral interferometry or frequency comb interferometry. This allows for rapidly measuring spectral interferometry, in particular if knowledge is present a priori at least approximately on the size of the optical path difference in the spectral interferometer, for example if the measured object is an almost planar, small object or a thin layer, and if said optical path difference is limited to the rather microscopic region. For hyperspectral imaging or for three-dimensional measuring methods—such as the chromatic confocal method—, however, the suggested light source is not applicable or applicable with limitations only.
Further, from prior art there are known methods and corresponding devices for multispectral or hyperspectral imaging using tunable light sources (so-called “swept sources”). These methods and devices, however, are very slow and are not suited for “real time” imaging or measurements, respectively. The article “Simultaneous three-dimensional step-height measurement and high resolution tomographic imaging with a spectral interferometric microscope” by D. Mehta et al. in Applied Optics, Vol. 41, No. 19, pp 3874-3885, 2002 describes a tunable light source having liquid crystal based Fabry-Perot interferometers which normally provides quasi-monochromatic light only.
A further disadvantage of the prior art methods and devices for multispectral or hyperspectral imaging is the reduced signal-to-noise ratio because not all of the spectral components are simultaneously in optical and/or data connection with the imaged or analysed object. Moreover, these methods lack flexibility as regards selection of the parameters for hyperspectral analysis.
It is an object of the present invention to provide improved methods and devices for multispectral or hyperspectral imaging, for detecting structural and topographic information on an object or the distance to an object by means of spectroscopy, or for spectral modulation of radiation across space and time with dynamics reduction. In particular, it is an object of the invention to obtain multispectral or hyperspectral information on an object over the whole surface, wherein spatial resolution and/or the signal-to-noise ratio and/or the information acquisition speed are improved.
These objects are achieved by an illumination device for generating multispectral or hyperspectral illuminating light having an addressable spectrum according to claim 1, a multispectral or hyperspectral imaging device according to claim 7, a multispectral or hyperspectral measuring device according to claim 12, a method of generating multispectral or hyperspectral illuminating light according to claim 13, and a method of multispectral or hyperspectral imaging and/or of distance measurement and/or topographic measurement according to claim 16. Preferred embodiments are subject of the subclaims.
According to a first aspect of the invention, there is provided an illuminating device for generating multispectral or hyperspectral illuminating light having an addressable spectrum. The illuminating device comprises a multispectral light source and a modulator, or modulation device, for temporal modulation of the individual spectral components of the light emitted by the multispectral light source, wherein the temporal modulations of the individual spectral components are different from one another. In particular, the individual spectral components are modulated with mutually different modulation frequencies, modulation frequency ranges and/or modulation sequences.
The multispectral light source can comprise    (i) at least one light source having a continuous, quasi-continuous, or frequency comb spectrum, and (first) wavelength-dispersive means for spectral splitting of the light emitted by the light source in a plurality of spatially separate spectral components having mutually different wavelengths λ1, λ2, . . . , λn or wavelength bands, respectively; and/or    (ii) an assembly or array of spatially separate monochromatic or quasi-monochromatic light sources which are configured to emit light having predetermined emission wavelengths λ1, λ2, . . . , λn or emission wavelength bands, differing from one another.
Accordingly, the modulator can comprise    (i) at least one electrically controllable spatial light modulator which is configured to modulate the individual spectral components in terms of time; or    (ii) a plurality of electronic control modules assigned to the individual monochromatic, quasi-monochromatic light sources.
The illuminating device further comprises optical means which are configured to combine the individual, modulated spectral components such that the individual, modulated spectral components substantially spatially overlap each other so as to form the multispectral or hyperspectral illuminating light. The optical means can comprise (second) wavelength-dispersive means (e.g. diffraction gratings, prisms, etc.). The second wavelength-dispersive means are configured to compensate for the spatial separation of the individual spectral components or the angular difference of the individual spectral components, respectively.
The generated multispectral light having an addressable spectrum can be employed in a variety of applications, in particular for medical and measuring applications. In the present application, the term “light” is understood as any electromagnetic radiation, e.g. any electromagnetic radiation within the visible spectral range (VIS), the ultraviolet spectral range (in particular in the extreme (EUV), deep (DUV) or weak (UV) ultraviolet), the infrared spectral range (in particular in the near (NIR), mid (MIR) or far (FIR) infrared, in the terahertz spectral range and/or in the X-ray range.
The multispectral light source can comprise at least one light source having a continuous, quasi-continuous spectrum, or a frequency comb spectrum and wavelength-dispersive means. The light source can be, for example, a line source or a point source, e.g. at least one light emitting diode (LED), a white light diode, a super luminescent diode (SLD), etc. Likewise, the light source can be a light source having a quasi-continuous spectrum, e.g. a light source comprising a plurality of individual sources having different, spatially overlapping spectral lines or spectral ranges, respectively. The light source can further comprise at least one frequency comb laser or a super luminescent diode (SLD) with downstream Fabry-Pérot interferometer (FPI). The source of electromagnetic radiation, in particular for the mid infrared range, can be formed as a synchrotron radiation source.
The individual spectral components can be spatially separated from each other by means of wavelength-dispersive means. The wavelength-dispersive means are preferably configured and arranged such that the light emitted by the light source is spectrally split such that the individual spectral components having mutually different wavelengths or wavelength band ranges are spatially separate from each other in a two-dimensional or three-dimensional region (e.g. in a predetermined plane). In this three-dimensional region, there is at least one lateral wavelength axis, or spectral axis, or a predetermined, laterally structured range of intensities having different wavelengths. These latter spectral ranges can be located substantially perpendicular with respect to the wavelength axis. The wavelength-dispersive means can be, for example, diffraction gratings, prisms, Fabry-Pérot interferometers, etc.
At least one electrically controllable spatial light modulator can be arranged in the two-dimensional or in the three-dimensional region. The spatial light modulator (SLM) is configured to accomplish time modulation of the individual, spatially separated spectral ranges. For example, a different time modulation of the individual spectral ranges or spectral components, respectively, can be obtained in lateral direction (along the wave axis), by varying the degree of reflection and/or the degree of transmission, and/or the degree of absorption of the electrically controllable spatial light modulator using a predetermined controlled time modulation (periodic or aperiodic). In other words: time modulation of the spatially separated, differing spectral ranges is accomplished by means of space-time modulation, or control, of the spatial light modulator.
The term “spatial light modulator” covers all kinds of spatial modulators, including spatial modulators for light in the visible, the infrared, the ultraviolet or terahertz range. The spatial modulator can, for example, modulate the degree of reflection and/or the degree of transmission, and/or the degree of absorption of the incident light. Suitable spatial modulators are, for example a digital mirror device (DMD), a liquid crystal display (LCD), a liquid crystal on silicon (LCoS). The DMD is preferably used as the spatial modulator. DMDs have very high light efficiency as regards the light modulated elements, which is better than in the case in which the DMD is used as an amplitude diffraction grating.
Multispectral light having a plurality of spatially separated spectral components can also be generated by means of an assembly or an array of monochromatic or quasi-monochromatic, discrete light sources which are arranged so as to be spatially separated. The individual light sources within the assembly or the array are configured to emit light, or light rays, having mutually different emission wavelengths λ1, λ2, . . . , λn or emission wavelength bands. The individual discrete light sources can be controlled directly by means of electronic control modules assigned to the respective individual light sources, so as to time-modulate the intensity of the individual emission wavelengths or emission wavelength bands. In this case, the multispectral light source can comprise an assembly or an array of LEDs, SLDs, laser diodes, etc. each emitting light having a predetermined wavelength or light in a predetermined narrow spectral range.
The number of spectral components can be selected in dependence on the desired application of the illuminating device. If the illuminating device is used in a measuring device for detecting the 2D or 3D profile of an object and/or for measuring the distance to an object, the number of individual spectral components, or the distance of the individual spectral lines, respectively, can preferably be selected such that each spectral line, or each spectral component, addresses, or corresponds to, a distinct depth in the object space.
The device for generating multispectral or hyperspectral illuminating light having an addressable spectrum can further comprise means which are configured to spectrally filter the light emitted by the light source having the continuous, quasi-quasi-continuous, or frequency comb spectrum, or the light emitted by the individual monochromatic or quasi-monochromatic light sources. The means for selective spectral filtering can be configured to selectively filter out, or cut-off, individual spectral components and/or spectral ranges from a wide spectral range. The means for selective spectral filtering can be one or more fixed or electrically controllable spectral filters and/or light traps and/or absorbers, or other optical components.
The means for elective spectral filtering can be connected downstream the light source having the continuous, quasi-continuous, or frequency comb spectrum, or the individual monochromatic or quasi-monochromatic light sources. Further, the means for selective spectral filtering can be integrated in the spatial modulator or in the electronic control modules (e.g. as part of the controller).
In particular, the spatial light modulator can be configured to perform selective adjustment, or selective filtering, of the spectral range of the continuous, quasi-continuous, or frequency comb spectrum light source. For example, the spatial light modulator can be configured to filter out, or cut-off, individual spectral components and/or spectral ranges (e.g. by deflection toward a light trap). In one embodiment, the spatial modulator can serve as a variable (e.g. relatively narrow-band) spectral filter.
Spectral filtering of the spectral range allows for adaptation of the multispectral illuminating light to the purpose of the respective application. This is advantageous, in particular when medical applications are concerned where living objects are examined. Since the multispectral light can be exactly adapted to the subject to be illuminated, or examined, needless exposure of the subject to be illuminated to radiation which might be harmful to the tissue can be avoided or minimized, in particular where a synchrotron light source or another source of potentially injurious or harmful electromagnetic radiation is used as the light source. In contrast, conventional attempts of spectral measurement, in particular in the MIR range, use a spectrum as complete as possible for illumination in order to detect any possible spectral effect.
In one embodiment, the spectral distributions applied are predetermined, or specifically thinned out, by the means for selective spectral filtering. Those spectral components which most certainly do not address any tumor markers or other relevant markers, and which do not contribute to acquiring information can thus be excluded or filtered out. This is advantageous in that the exposure (e.g. if an infrared radiation source is used, the thermal load which is generated in the tissue due to high absorption of water) of the illuminated subject (e.g. an organ) is reduced. Further, the radiation energy can be increased in those spectral ranges where spectral tumor markers or other relevant markers are present. This allows for obtaining a higher signal-to-noise ratio of the measuring signal, or the spectral information obtained, respectively, leading to higher velocity and accuracy of the measurement, or imaging, respectively. Since the detected signals which are processed for obtaining information on the illuminated subject contain less spectral components, the velocity of measurement or imaging, respectively, can be increased, too.
Further, it is possible to have a frequency spread in the signal space of the spatial light modulator and/or of a detector and/or of a digital signal processing device. Therein, spreading in the frequency range of the modulation can be higher than in the range of the physical wave numbers.
For example, a plurality of different narrow band signatures (e.g. tumor marker narrow band signatures) can be addressed one after the other (e.g. 3×5 different narrow bands instead of 30 potential narrow bands at the same time). With these narrow bands, or spectral components, addressed via SLM elements, a “quasi frequency spread” in the low-frequency frequency range of the electric/digital signal processing is performed by means of the SLM elements, as described below. Thus, optimal use of the signal processing and the signal transmission capacities provided by the system and the components is possible. Therein, the physical wavelength/wave number of the narrow band radiation remains unchanged. A low-frequency spread—typically far below 10 KHz—is accomplished only within the signal space of SLM, detector and digital data processing. Further, it is possible to provide for a frequency spread for ensuring optimal use of signal transmission and signal analysis when known components and systems are used.
The suggested solution of selective filtering of individual spectral ranges and/or spectral components and of the frequency spread can be used independently of the above described time modulation of the individual spectral components. A solution of this kind is advantageous in particular when the illuminating device is used in the medical field for illuminating living subjects (e.g. within the framework of therapeutic measures, such as during a surgical intervention) where only the information required by the surgeon (cutting off tissue yes/no) must be obtained with high accuracy and at the required speed (few minutes at maximum, e.g. less than 20 minutes for the total time of all diagnostic measures during a surgical intervention).
The suggested solution of selective filtering of individual spectral ranges and/or spectral components, and of frequency spread (also referred to as selective spectral frequency spread and or selective frequency spread) is particularly advantageous if a brilliant source of high spatial coherence, in particular also in the mid infrared range (MIR) is used as the source. Examples of such sources include, e.g. synchrotron radiation sources having very high-energy, brilliant radiation, in particular also in the mid infrared (MIR), or laser batteries of spectrally finely tuned lasers, which radiate also in particular in the mid infrared (MIR).
In one embodiment, the radiation from a brilliant source of high spatial coherence is spectrally split very finely, for example using a device for spectral splitting of electromagnetic radiation (e.g. with a high-resolution grating spectrometer) in the MIR. Suited means are used for selecting specific spectral ranges which contribute to acquisition of information, e.g. spectral ranges which address known tumor markers. Each spectral range can comprise a plurality of individual spectral components (e.g. as narrow spectral bands). Those spectral components and/or spectral ranges which are not selected can be filtered using suitable means for spectral filtering, and can be excluded from propagation toward the subject to the best possible extent. For example, spectral components and/or spectral ranges not selected can be directed toward light traps or can be “blanked”.
A spatial light modulator which is preferably optimized for the MIR can be assigned to each of the selected ranges. The spatial light modulators usually are spatially separated from each other in the area (plane) of spectral splitting of the device for spectral splitting of electromagnetic radiation. As an alternative, spatially separated ranges of one single spatial modulator can be used. The illuminating device can comprise optical means which are configured to deflect, or to image, the light from the spatially separated spectral ranges onto the different ranges of the spatial light modulator.
Spectral splitting (resolving power) of the device for spectral splitting of electromagnetic radiation can be chosen such that each spectral component can be assigned its unique harmonically oscillating element of the spatial light modulator. As an alternative, a group of elements of the spatial light modulator can be assigned to each spectral component. Each spectral component is modulated by the element of the spatial light modulator assigned thereto, or by the group of elements of the spatial light modulator assigned thereto with the frequency assigned to the element. Accordingly, the spread in the frequency range of the modulation can be much greater than in the range of the physical wave numbers.
The subject under examination (e.g. a patient's organ during a surgical intervention) is radiated with the addressed multispectral radiation. The irradiated subject is viewed using a detector which is preferably designed for the MIR. Information on the subject under examination can be acquired through frequency analysis of the detected signal (e.g. a lock-in detection).
Specific selection and addressing of certain spectral ranges and/or spectral components allows for improving the signal-to-noise ratio and for reducing the signal processing time. Further, exposure of the illuminated subject can be reduced as only a relatively small portion of the radiation from the source contributing to information acquisition reaches the subject. Therein, the brilliant source can run at “full load”.
Another solution is in time sequential wavelength tuning by means of SLM in a device for high-resolution spectral splitting. In this case, only one single narrow band radiation, or one single spectral component, is applied. As this solution requires an especially high luminosity of radiation, this solution is usually inferior to the solution using a plurality of narrow band lines, or a plurality of spectral components, as regards the signal-to-noise ratio that can be achieved.
Preferably, the light source, or the individual monochromatic or quasi-monochromatic light sources have as high luminosity as possible, thereby improving spectral resolution.
Time modulations of the individual spectral components can differ in their modulation frequency, modulation frequency range, modulation sequence and/or the starting phases of the modulations. For example, the time modulations of the individual spectral components can differ in their basic frequency or in their (relatively narrow) basic frequency range (wherein harmonic waves can occur) and/or in their modulation sequence.
The time modulation of the individual components can be periodic or aperiodic, wherein harmonic waves can occur, too. In particular, the individual spectral components can be time modulated with harmonic oscillations, with the modulation frequencies lying in the range of below 10 MHz, preferably in the range of 10 Hz to 10 KHz.
Modulation may comprise a frequency chirping, i.e. a time variation of the modulation frequency of the preferably harmonic oscillations or modulations, respectively. This can be achieved by a time variation of the modulation frequency of the SLM or of the modulation frequency of the individual control modules, respectively.
As an alternative or in addition, the modulation device can be configured to differently determine, select, or vary, respectively, the starting phases of the modulations of the individual spectral components. Further, the modulation device can be configured to image, in a nonlinear manner, the starting phases of the modulations of the individual spectral components in the signal detected by a rasterized detector. The variation of the starting phases can differ from modulation frequency to modulation frequency, and can occur in a linear or in a nonlinear manner. Likewise, stochastic or random variation is possible. Preferably, variation of the starting phases is time-invariant.
Preferably, the variation of the starting phases is not random, but controlled. For example, the phase shifts (with respect to the starting time or reference time) can be continuously falling or continuously rising. Preferably the phase shifts are nonlinear, preferably square, depending on the modulation frequency. Preferably, the magnitude of the phase angle changes only little. In this case, the processing power required for calculating the spectra, or for spectral discrimination mostly tends to a minimum. Further, optimal results can be obtained, i.e. the calculated spectra have minimum error.
Further, the modulation device can be configured to vary the amplitudes of the preferably harmonic modulations of the individual spectral components in terms of time. In particular, the modulation device can be configured to perform an apodization wherein normally the amplitudes in the harmonic oscillations preferably decrease after the start of the modulation (e.g. after the beginning of the oscillations at the SLM elements).
As explained below, the above solutions allow for reduction of the demands on the dynamics of the detector because the signals recorded usually have lower mid signals).
If the illuminating device is, for example, used in a device for multispectral or hyperspectral imaging for illuminating an object, the time signal for each pixel is obtained pixel-by-pixel from an image batch of the camera which is read out for a predetermined period of time. As there are many pixels, a corresponding number of time signals are recorded. In particular, the pixels of the camera at any time record a sum signal as overlap of all the individual signal oscillations generated by the spatial light modulator or by the control modules. Herein, an individual frequency f_i is assigned to a physical wave number band delta_kb_i which was created by spectral splitting, by means of special control of the spatial light modulator or of the control modules, for example by light of a wave number band delta_kb_i being incident on an SLM element I which oscillates with the frequency f_i in a computer-controlled manner.
In dual-beam interferometry with multispectral light, the chirp effect means that the optical path difference of zero is obtained for radiation of different wave numbers for different reference mirror positions. This means that there is a varying phase difference above wave number k which is noticeable in the two-beam interferogram by a frequency change. In some cases, this frequency change is clearly noticeable in the two-beam interferogram and is referred to as chirping or chirp effect. In infrared Fourier transform spectroscopy, the phase difference above wave number k is numerically calculated from a short double-sided interferogram using an FFT, and is used for correction.
According to an aspect of the invention, however, a predetermined phase difference is deliberately imposed in order to reduce the mid peak in the sum signal (corresponding to the interferogram). This phase difference is thus known a priori. Accordingly, phase correction in the calculation of spectra can be carried out using the a priori known (predetermined) phase difference.
In a preferred embodiment, a signal having a chirp effect (=a chirped signal) is generated within the period of time, said signal being detected in a respective pixel j by means of an image batch taken by a camera (a rasterized detector). When observing, for example, the case of harmonic oscillations of the signal intensity over the period of time, the chirp effect means that each harmonic oscillation (in an assembly of oscillations) has a frequency-depending phase shift phi(f_i) at a reference time. This time can be the illumination starting time t_0 (e.g. upon activation of the SLMs in the measuring process). Prior to the starting time t_0, there is no light flux from the SLM or the electronically controllable multispectral light source, whereby illumination of the camera is effected starting at the starting time t_0 with harmonic oscillations with the different frequency f_i and with different phase shift phi(f_i) (=different starting phase). During the complete measuring time, i.e. illumination time of the rasterized detector, the harmonic oscillations are preferably at no time all in-phase.
Signals (sum signals as overlap of a plurality of harmonic oscillations) with chirp effect are, however, only one of several possibilities of limiting dynamics of a signal.
In addition, or as an alternative, the starting phase of the individual harmonic oscillations can be selected, or determined, to be mutually different (accordingly, the starting phase value of zero merely is an exclusion). The starting phase of the individual harmonic oscillations can be continuously falling or rising, linearly or nonlinearly, or it can be randomly distributed. Therefore, there are sum signals exhibiting no chirp effect at all, or having no pronounced chirp effect. These signals, or sum signals can still have no pronounced maximum, and are thus suited for reducing the demands on the modulation of a detector.
In one embodiment, the individual spectral components can be modulated over the whole period with the respective invariable frequency assigned to them once (e.g. by means of a spatial light modulator). This frequency differs from spectral component to spectral component and is mathematically unambiguous. A narrow wave number band delta_kb_i in the spectrum of the multispectral illumination bundle can be assigned to each spectral component. Each spectral component has the modulation frequency f_i at the SLM element i, assigned exclusively to the respective spectral component. A sum signal is created by overlap of the individual oscillations, generated by the differently oscillating elements of the spatial light modulator, with the overlap finally occurring on/at each of respective pixels of the camera. Although the frequencies of the individual harmonic, summed-up oscillations remain constant over the period of time, the detected time signal exhibits the effect of a variation in terms of frequencies of the individual modulations in the sum signal, wherein an increase or decrease of the frequency of the modulations (chirping) can often be observed in the sum signal. By means of specific interfering with respect to the starting phases in harmonic signals, modulation with dynamics reduction can be obtained.
Through frequency chirping (phase difference in the time domain) or through differing (including random) starting phases of the frequency encoding, optimum adaptation of the radiation intensity to the dynamics of the detection system (usually a camera, e.g. a CCD camera, or an alternating-light detector for the infrared spectral range) can be achieved, wherein the advantage is greater for an alternating-light detector since there is no need to detect a constant component in the alternating signal which also makes demands on the given scope of dynamics. This can improve the signal-to-noise ratio and reduce the measuring time, which constitutes a great advantage in particular where medical applications are concerned. This can, for example, make sure that during diagnosis of tissue during a surgical intervention, thermal stress of the tissue is not exceeded. In case of aperiodic modulation, control of the spatial light modulator is preferably synchronized with the detection system.
The modulation device can further be configured to linearly change the modulation frequency of the individual spectral components. The modulation frequency can, for example, be a linear function of the wavelength, or of the focus wavelength of a wavelength range, or of the wave number. In a spatial light modulator, the modulation frequency can, for example, change linearly along the spectral axis (the wave axis, or the wave number axis (in the k-space), e.g. in column direction or in row direction of the spatial light modulator. Preferably, the modulation frequency changes linearly with the way number. This allows for algorithmic easy evaluation which is quick and requires comparatively low computing power. It is, however, possible to change the modulation frequency non-linearly.
One advantage of the suggested illuminating device is the high flexibility thereof. For example, the parameters of modulation (time modulation and/or spatial modulation) can be flexibly selected and changed with a “mouse-click” depending on the respective application. The modulation frequency is freely selectable with respect to the physical wavelength of the electro-magnetic radiation. For example, the modulation frequency can be “inverted” with respect to the physical one. Further, the modulation frequency can be freely selected. In particular, the modulation frequency can be freely shifted: first, shortest wavelength with transformation to smallest modulation frequency, then shortest wavelength with transformation to greatest modulation frequency. In general, the positions of same modulation frequency can likewise be freely selected and can “travel”, for example, in lateral direction (e.g. along the wave axis, or wave number axis of the spatial light modulator, or along the array of individual light sources with associated control module). This allows for better averaging of the obtained signal. The change of the modulation frequency can further be effected depending on the wavelength ratio or the wave number ratio such that the spectrum need not be transformed (stretched or compressed).
Further, the plurality of possible encodings, in particular in conjunction with suitable evaluation, allows for discrimination of a large number of spectral channels. A further advantage lies in the fact that use of movable parts is not required, which leads to a further increase of precision and accuracy of the device. Furthermore, the illuminating device can have a compact structure.
With the illuminating device according to the invention, it is possible to provide time-space-modulated electromagnetic radiation for a plurality of imaging and measurement methods and devices, such as for multispectral or hyperspectral cameras, for chromatic-confocal or interferometric, in particular also for spectral-interferometric measuring methods and devices, fluorescence microscopy, multi-photon microscopy, etc. With the time-space-modulated light, for example, spot light source illumination can be achieved within the object space of a chromatic-confocal measuring arrangement, thereby obtaining information on the 2D and 3D profile of an examined object via chromatic depth scanning.
According to a second aspect of the present invention, a device for multispectral or hyperspectral imaging is suggested, comprising
the illuminating device for generating multispectral or hyperspectral illuminating light having an addressable spectrum according to the first aspect of the invention;
an image acquisition device which is configured to record a sequence of two-dimensional images of an object illuminated with the multispectral light, comprising at least one two-dimensional rasterized detector having a plurality of detector elements which are configured to detect the intensity of at least part of the light coming from the object (reflected or transmitted light, luminescence or fluorescence light);
an image evaluation device which is configured to determine the shares of the individual spectral components (in the acquired images, which have been, for example taken as an image batch) by a pixel-by-pixel analysis of the time variation in the intensity detected by each of the detector elements, and to form a multispectral or hyperspectral image of the object based on the determined shares of the individual spectral components.
In particular, the shares of the individual spectral components can be determined for each detector element by an analysis (e.g. a frequency analysis) of the detected intensity profile l(t), as well as on the basis of information on the used encoding and modulation, respectively, of the individual spectral components, and a multispectral or hyperspectral image of the object can be generated. The information on the used encoding or modulation, respectively, can comprise, for example, information on the assignment of the individual frequency components to the individual spectral ranges, e.g. via a look-up-table (LUT).
As described above, the modulation device can be configured to modulate the individual spectral components with different basic frequencies or basic frequency ranges and/or with different modulation sequences. The image evaluation device can be configured to perform, for each detector element, a frequency analysis of the time variation of the detected intensity and a modulation-wavelength conversion by means of an (at least approximately known a priori) assignment (via an LUT, for example) of the determined modulation frequencies to a certain basic frequency or a certain basic frequency range and/or a certain modulation sequence of the individual spectral ranges.
For example, the analysis carried out by the image evaluation device can be, or comprise, a pixel-by-pixel Fourier analysis, in particular an FFT analysis, a wavelet analysis, a lock-in detection, or another suitable analysis of the detected intensity profile, or the detected time-variation of the intensity. The image evaluation device can further be configured to perform a correlation of the modulations determined using the detected intensity with the modulations of the spectral components accomplished by the modulation device in order to discriminate the individual spectral components.
The device for multispectral or hyperspectral imaging can further comprise a synchronization device which is configured to synchronize the modulation of the individual spectral components by means of the modulation device and image acquisition by means of the image acquisition device.
This way, especially where aperiodic encodings are concerned, synchronization in terms of time can be performed between the modulation device for multiplex encoding, in terms of frequency, of the spectral components of the multispectral light, or of the spectral distribution of one of more light sources, respectively, and the image acquisition device (in particular the two-dimensional detector). If a priori information are available on the encoding used or in case of periodic encoding, synchronization can be omitted. In particular, start of the SLM modulation, or SLM control or control of the individual electronic control modules, respectively, can be synchronized with the start of image acquisition. The synchronization allows for improving stability, accuracy, and image quality of the multispectral or hyperspectral imaging device.
The device for multispectral or hyperspectral imaging can further comprise a plurality of Fabry-Pérot filters for demodulating the spectral wavelets in the frequency space.
The device for multispectral or hyperspectral imaging can further comprise a memory for temporarily and/or permanently storing the detected sequence of two-dimensional images; and/or for temporarily and/or permanently storing the determined multispectral or hyperspectral image. The image evaluation device can further comprise at least one processor (e.g. a graphics processor) which is configured to carry out the required mathematical operations for generating the multispectral or hyperspectral image of the object. The processor can further be configured to render the generated, and if applicable stored image of the object for display (preferably as a 3D picture) on a display device (e.g. a computer monitor, camera monitor, etc.). The display device can be part of the device for multispectral or hyperspectral imaging.
The imaging device and the image evaluation device can be integrated with a camera module, e.g. in a so-called “smart camera”, i.e. a camera having computing capacity, or “on-chip intelligence”. In dependence on the application, the rasterized detector can be a CCD camera, a CMOS camera, a bolometer array for MIR spectral range, InGaAs camera for NIR spectral range, etc. for example.
One advantage of the suggested device for multispectral or hyperspectral imaging is that all spectral components are in optical contact with the object substantially at the same time. This allows for considerably reducing the signal-to-noise ratio. A further advantage resides in the massive parallelizability of data acquisition and data processing, which is possible in particular thanks to the use of specialized hardware (such as smart cameras, graphics processors, etc.) and software. Further advantages include the advantages already mentioned in connection with the illuminating device, such as high flexibility, decoupling of the modulation frequency from the modulated physical wavelength of the electromagnetic radiation, the large number of possible encodings and spectral channels.
According to a third aspect of the invention, there is provided a multispectral or hyperspectral measuring device for distance measurement for and/or for topographic measurement of an object by means of spectrometry using an illuminating device according to the first aspect and a device for multispectral imaging according to the second aspect. The measuring device for distance measurement for and/or for topographic measurement further comprises—depending on the application—a chromatic-confocal system, a chromatic triangulation system, a spectral interferometer, a fluorescence microscope, or a multi-photon microscope. Thereby, color data, data regarding the distance to one or more objects, the 2D or 3D profile of an object, topography of an object, distribution of certain materials and/or structures within an object, etc. can be obtained.
Preferably, the light supply in the above devices according to the first to third aspects is accomplished at least partly via light conductors. For example, the light can be guided to the spatial light modulator and/or to the other optical components (such as the chromatic-confocal system, microscope lens, etc.) at least partly vial light conductors. Preferably, a multi-bundle (e.g. 8 bundles) to single-bundle coupling and/or a multi-fiber (e.g. 8 fibers) to single-fibre coupling is performed. Light supply or light guiding, respectively, between the individual optical components can likewise be accomplished by means of adapted optical deflection devices, comprising at least a mirror, beam splitter, a lens, and/or other optical elements.
According to a fourth aspect of the invention, there is further suggested a method of generating multispectral or hyperspectral illuminating light having an addressable spectrum. The method comprises:
generating multispectral light; and
time-modulating the individual spectral components of the multispectral light (including in particular time-modulation of the amplitude of the individual spectral components and/or determination of the starting phase of the modulations) with modulation frequencies, modulation frequency ranges, and/or modulation sequences differing from each other, respectively, wherein                (i) the generation of multispectral light comprises spectral splitting of the light emitted by a continuous, a quasi-continuous, or a frequency comb light source in a plurality of spatially separate spectral components having mutually different wavelengths λ1, λ2, . . . , λn or wavelength bands; and the time modulation of the individual spectral components is accomplished by means of an electrically controllable light modulator; or        (ii) the generation of multispectral light comprises the emission of light having a plurality of mutually different spectral components with mutually different emission wavelengths λ1, λ2, . . . , λn, or emission wavelength bands from an arrangement or an array of a plurality of monochromatic or quasi-monochromatic light sources, and the time-modulation is accomplished by means of electronic control modules assigned to the individual light sources; and        wherein the method further comprises                    combining the individual modulated spectral components of the light emitted by the multispectral light source such that they substantially spatially overlap each other so as to form the multispectral or hyperspectral illuminating light having an addressable spectrum.                        
As explained in connection with the illuminating device according to the first aspect of the invention, time modulation of the individual components can be periodic or aperiodic. Time modulation can comprise, e.g., frequency chirping or a variation of the starting phases, which is different from modulation frequency to modulation frequency, but is preferably time-invariant. Variation of the starting phases is preferably non-linear, depending on the modulation frequency; in particular it is preferably square. Further, time modulation can comprise a linear change of the modulation frequency of the individual spectral components.
Further, modulating can comprise changing the amplitude of the modulations of the individual spectral components. For example, an apodization can be carried out.
Further, the method of generating multispectral or hyperspectral illuminating light preferably comprises selective spectral filtering of the light emitted by the light source having the continuous, quasi-continuous or frequency comb spectrum, or of the light emitted by the individual monochromatic or quasi-monochromatic light sources, as described in connection with the illuminating device according to the first aspect of the invention.
According to a fifth aspect of the present invention, there is further suggested a method of multispectral or hyperspectral imaging and/or of distance and/or topographic measurements of an object. The method comprises:
generating multispectral or hyperspectral illuminating light having an addressable spectrum according to the method of the fourth aspect of the present invention;
illuminating the object with the multispectral or hyperspectral illuminating light;
detecting a time sequence (image batch) of two-dimensional images of the illuminated object using a two-dimensional rasterized detector having a plurality of detector elements, the detector being configured to detect the intensity of at least part of the light coming from the object;
determining the shares of the individual spectral components by means of a pixel-by-pixel analysis of the time variation of the light intensity detected by each detector element, and generating a multispectral or hyperspectral image of the object based on the determined share of the individual spectral components.
Determining the shares of the individual spectral components can comprise a frequency analysis of the time variation of the detected intensity, or of the respective detector signal, and a modulation wavelength conversion by means of an (at least approximately a priori known) assignment of the determined modulation frequencies to a specific basic frequency, or to a specific basic frequency range and/or to a specific modulation sequence of the individual spectral ranges.
Determining the shares of the individual spectral components can comprise a pixel-by-pixel Fourier analysis or a wavelet analysis of the detector signals, or of the intensity detected by each detector element, respectively, and/or a correlation of the modulations determined using the detected intensity with the modulations of the spectral components performed by the modulation device.
Preferably, the method further comprises synchronizing of modulation of the individual spectral components and detection of the time sequence (image batch) of two-dimensional images.
The methods and devices according to preferred embodiments of the present invention can exhibit the following advantages:
In comparison to the solutions using tunable light source (the so-called “swept source” solution) the solution according to the invention is advantageous in that all the spectral components, or a plurality of spectral components are on optical contact with the object at the same time, which leads to higher light efficiency and improved signal-to-noise ratio. A further advantage related to signal processing resides in the massive parallelizability which is possible in particular thanks to the use of so-called smart cameras or specialized graphic processors. A further advantage is the fact that all the optical elements (e.g. dispersive optical elements, spatial modulators, etc.) can be optimally used as regards the optical efficiency thereof.
Further advantages of the devices and methods according to preferred embodiments of the present invention can comprise:                Very high flexibility of the parameters of spectral and spatial resolution in multispectral and hyperspectral imaging. The parameters can be changed by “mouse click”.        Very high data rate thanks to multiplexing and highly parallelizable signal evaluation;        3D-detection is also possible with high flexibility and in a very wide range of scales;        Very high scalability, such that nano-, micro-, and macro-applications are possible, too;        Possibility of miniaturization;        Simple realization, also for mass products;        High detection speed, reliability, and specific in multispectral and hyperspectral imaging. For example, a “quasi time-resolved” detection of tissue is possible in wide spectral ranges (VIS, NIR, MIR, FIR, Terahertz range). Thus, changes in the tissue structure (e.g. changes of the blood supply of the tissue during medication, monitoring in intensive care) over the time can be visualized in real time or quasi-real time for the physician in a two-dimensional image of the organ (e.g. after a transplantation). Further very good adaptation to a wide field of diagnostic tasks is possible in ambulant or hospital medical treatment.        
The suggested devices and methods of generating multispectral or hyperspectral illuminating light for multispectral or hyperspectral imaging and measurement are in particular suited for rapid, full area multispectral or hyperspectral 2D or 3D imaging and examinations. Further, the suggested devices and methods are suited for imaging examinations through detection of the reflected, transmitted, absorbed or scattered light, or of the luminescence light (including fluorescence light) emitted by the object.
The fields of application of the invention comprise:                Multispectral or hyperspectral cameras, including cameras for mass production;        Medical applications for ambulant or hospital patients' diagnostics, e.g. in dental diagnostics, diagnosis of tumor diseases, in particular for rapid and reliable diagnosis, or detection of tumors (status analysis) during surgical interventions;        Dental technology, in particular 3D dental technology on patients (in vivo) with spectral analysis of the tooth material so as to improve aesthetics of dental prostheses by optimal adaptation of the dental prostheses to the natural color of the teeth.        Applications in measuring technology in material science, e.g. for welding line inspection, 2D or 3D detection of object profiles and topographies with simultaneous detection of color, e.g. for detection of traces of powder);        Production monitoring (e.g. for rapid high-resolution multispectral and hyperspectral imaging of textiles, colors, varnish, resins, etc.);        Monitoring of food production (e.g. examination of the degree of ripeness of fruit and vegetables, of freshness of animal products, etc.);        Animal rearing;        Medical research;        Protection against forgery (e.g. for examination of bank notes, paintings, etc.);        Analysis of objets d'art (e.g. for analysing varnish layer succession);        Rapid and high-resolution 3D detection by means of spectral information in technical and medical applications, in particular for 3D-measuring technology, such as chromatic-confocal sensor technology, chromatic triangulation measuring technology and spectral-interferometric applications;        Microscopic analysis.        
Throughout the Figures same reference numbers are used to denote same or similar elements. Moreover, a list of reference numerals and corresponding explanations are provided in Table I.